Information processing apparatus, method of processing information, and storage medium

ABSTRACT

An information processing apparatus includes circuitry to register at least a first viewpoint and a second viewpoint designated next to the first viewpoint as viewpoints in a full-view spherical image with an order designated by a user, set a transition path of the viewpoints from the first viewpoint to the second viewpoint by interpolating between the first viewpoint and the second viewpoint in the full-view spherical image, generate a first partial image having a center that matches the first viewpoint, and a second partial image having a center that matches the second viewpoint, and play animation by sequentially displaying the first partial image and the second partial image with the order designated by the user while transiting the viewpoints from the first viewpoint to the second viewpoint along the set transition path.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority pursuant to 35 U.S.C. § 119(a) toJapanese Patent Applications Nos. 2016-233059, filed on Nov. 30, 2016,2017-037632, filed on Feb. 28, 2017, and 2017-185161, filed on Sep. 26,2017 in the Japan Patent Office, the disclosure of which is incorporatedby reference herein in its entirety.

BACKGROUND Technical Field

This disclosure relates to an information processing apparatus, a methodof processing information, and a storage medium.

Background Art

Lately, full-view spherical cameras or omnidirectional cameras are usedin many situations, in which a full-view spherical image oromnidirectional image covering an omnidirectional scene is captured, andthe full-view spherical image can be viewed using a dedicated viewer.The dedicated viewer of the full-view spherical image is an applicationthat maps the full-view spherical image on a surface of a virtual threedimensional object to generate a three dimensional model, and performs aprojective transformation to a partial region of the three dimensionalmodel of the full-view spherical image based on a display-range changingoperation by a user such as pan, tilt, and zoom to display a twodimensional image.

When the user wants to find an area of interest in the full-viewspherical image by using conventional dedicated viewers, the user needsto perform a manual operation such as pan, tilt, and zoom for changing adisplay area of the full-view spherical image to search the area ofinterest by checking the entire of the full-view spherical image usingeyes of the user, which is not convenient for the user.

SUMMARY

As one aspect of the present invention, an information processingapparatus is devised. The information processing apparatus includescircuitry to register at least a first viewpoint and a second viewpointdesignated next to the first viewpoint as viewpoints in a full-viewspherical image with an order designated by a user, set a transitionpath of the viewpoints from the first viewpoint to the second viewpointby interpolating between the first viewpoint and the second viewpoint inthe full-view spherical image, generate a first partial image having acenter that matches the first viewpoint, and a second partial imagehaving a center that matches the second viewpoint, and play animation bysequentially displaying the first partial image and the second partialimage with the order designated by the user while transiting theviewpoints from the first viewpoint to the second viewpoint along theset transition path.

As another aspect of the present invention, a method of processinginformation is devised. The method includes registering at least a firstviewpoint and a second viewpoint designated next to the first viewpointas viewpoints in a full-view spherical image with an order designated bya user, setting a transition path of the viewpoints from the firstviewpoint to the second viewpoint by interpolating between the firstviewpoint and the second viewpoint in the full-view spherical image,generating a first partial image having a center that matches the firstviewpoint, and a second partial image having a center that matches thesecond viewpoint, and playing animation by sequentially displaying thefirst partial image and the second partial image with the orderdesignated by the user while transiting the viewpoints from the firstviewpoint to the second viewpoint along the set transition path.

As another aspect of the present invention, a non-transitory storagemedium storing one or more instructions that, when executed by one ormore processors, cause the one or more processors to execute a method ofprocessing information is devised. The method includes registering atleast a first viewpoint and a second viewpoint designated next to thefirst viewpoint as viewpoints in a full-view spherical image with anorder designated by a user, setting a transition path of the viewpointsfrom the first viewpoint to the second viewpoint by interpolatingbetween the first viewpoint and the second viewpoint in the full-viewspherical image, generating a first partial image having a center thatmatches the first viewpoint, and a second partial image having a centerthat matches the second viewpoint, and playing animation by sequentiallydisplaying the first partial image and the second partial image with theorder designated by the user while transiting the viewpoints from thefirst viewpoint to the second viewpoint along the set transition path.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the description and many of theattendant advantages and features thereof can be readily obtained andunderstood from the following detailed description with reference to theaccompanying drawings, wherein:

FIG. 1 illustrates an example of a schematic configuration of an imagecapture system including an information processing apparatus of anembodiment;

FIG. 2 illustrates an example of a hardware block diagram of theinformation processing apparatus of a first embodiment;

FIG. 3 illustrates an example of an appearance of the informationprocessing apparatus of the first embodiment;

FIG. 4 illustrates an example of a functional block diagram of theinformation processing apparatus of the first embodiment;

FIG. 5 illustrates examples of partial images;

FIG. 6 illustrates an example of an application screen when registeringa viewpoint;

FIG. 7 illustrates another example of an application screen whenregistering a viewpoint;

FIG. 8 illustrates an example of an animation during a preview playing;

FIG. 9 illustrates examples of partial images corresponding registeredviewpoints;

FIGS. 10A and 10B illustrate an example of an application screen whenediting a viewpoint;

FIGS. 11A and 11B illustrate an example of an application screen whensetting detail settings.

FIGS. 12A and 12B illustrate an example of an application screen whengenerating an animation;

FIG. 13 illustrates an example of a sequence diagram of processing whenregistering a viewpoint;

FIGS. 14A, 14B, and 14C illustrate examples of a viewpoint informationmanagement table;

FIG. 15 illustrates an example of a sequence diagram of processing whenplaying a preview;

FIGS. 16A and 16B illustrate an example of an application screen whenregistering a viewpoint in a second embodiment;

FIGS. 17A, 17B, 17C, and 17D illustrate examples of relationshipsbetween a selection of a direction instruction button and a transitiondirection from a first viewpoint to a second viewpoint;

FIG. 18 illustrates examples of preset data;

FIG. 19 illustrates an example of an application screen when preset datais used;

FIGS. 20A and 20B illustrate an example of an application screen whenpreset data is used;

FIGS. 21A and 21B illustrate another example of an application screenwhen preset data is used;

FIGS. 22A and 22B illustrate another example of an application screenwhen preset data is used;

FIG. 23 illustrates an example of parameters used for changing an imageexpression;

FIGS. 24A and 24B illustrate an example of a projection type;

FIGS. 25A and 25B illustrate another example of a projection type;

FIGS. 26A and 26B illustrate another example of a projection type;

FIG. 27 illustrate an example of an application screen when setting aprojection type;

FIG. 28 illustrates another example of a sequence diagram of processingwhen registering a viewpoint;

FIGS. 29A and 29B illustrate an example of a transition of animationbetween different projection types;

FIGS. 30 A and 30B illustrate another example of a transition ofanimation between different projection types;

FIGS. 31 A and 31B illustrate another example of a transition ofanimation between different projection types; and

FIG. 32 illustrates another example of a sequence diagram of processingwhen playing a preview.

The accompanying drawings are intended to depict exemplary embodimentsof the present invention and should not be interpreted to limit thescope thereof. The accompanying drawings are not to be considered asdrawn to scale unless explicitly noted, and identical or similarreference numerals designate identical or similar components throughoutthe several views.

DETAILED DESCRIPTION

A description is now given of exemplary embodiments of presentdisclosure. It should be noted that although such terms as first,second, etc. may be used herein to describe various elements,components, regions, layers and/or sections, it should be understoodthat such elements, components, regions, layers and/or sections are notlimited thereby because such terms are relative, that is, used only todistinguish one element, component, region, layer or section fromanother region, layer or section. Thus, for example, a first element,component, region, layer or section discussed below could be termed asecond element, component, region, layer or section without departingfrom the teachings of present disclosure.

In addition, it should be noted that the terminology used herein is forthe purpose of describing particular embodiments only and is notintended to be limiting of present disclosure. Thus, for example, asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. Moreover, the terms “includes” and/or “including”, when usedin this specification, specify the presence of stated features,integers, steps, operations, elements, and/or components, but do notpreclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof. Furthermore, although in describing views illustrated in thedrawings, specific terminology is employed for the sake of clarity, thepresent disclosure is not limited to the specific terminology soselected and it is to be understood that each specific element includesall technical equivalents that operate in a similar manner and achieve asimilar result. Referring now to the drawings, one or more apparatusesor systems according to one or more embodiments are describedhereinafter.

Hereinafter, a description is given of one or more embodiments of thepresent invention, but the present invention is not limited to the oneor embodiments to be described later. In this disclosure and drawingsreferred to below, the same reference numerals are used for commoncomponents and description thereof are adequately omitted.

FIG. 1 illustrates an example of a schematic configuration of an imagecapture system 1000 including an information processing apparatus 100 ofan embodiment. The image capture system 1000 includes, for example, atleast one information processing apparatus 100, and at least one imagecapture device 110 communicatively connected with each other. In anexample configuration illustrated in FIG. 1, the image capture system1000 includes three information processing apparatuses 100 a, 100 b, and100 c, and one image capture device 110, in which each of the threeinformation processing apparatuses 100 a to 100 c and the image capturedevice 110 are connected wirelessly. Further, each of the threeinformation processing apparatuses 100 a to 100 c and the image capturedevice 110 can be connected by wire.

The image capture device 110 can employ a full-view spherical camera oromnidirectional camera for capturing a full-view spherical image oromnidirectional image. In this disclosure, the full-view spherical imagemeans an image that is captured almost omnidirectionally around thefull-view spherical camera with one image capture operation. Thefull-view spherical image may not be an image exactly capturedomnidirectionally with 360 degrees. For example, the full-view sphericalimage captured by the image capture device 110 may not include an imagein a direction of several degrees or several tens of degrees from ahandle of the full-view spherical camera. The full-view spherical cameraincludes, for example, at least two fisheye lenses, at least two imagecapture elements, a controller, and a communication interface ashardware components.

The fisheye lens has an angle of view of 180 degrees or more. The angleof view of the fisheye lens indicates a range of image that can becaptured by a camera as a value of angle. The image capture element isdisposed at a position (imaging position) where the light is focused bythe fisheye lens, and converts an optical image formed by the focusedlight into image data of electric signal, and outputs the image data asthe electric signal. The image capture element can employ, for example,an image sensor such as complementary metal oxide semiconductor (CMOS)sensor, charge coupled device (CCD) sensor or the like.

The image capture device 110 includes, for example two fisheye lensesand two image capture elements, in which one of the two fisheye lenses(i.e., first fisheye lens) is disposed at one position, and another oneof the two fisheye lenses (i.e., second fisheye lens) is disposed atanother position opposing the one position, with which the first fisheyelens captures one hemispherical image having an image capturing azimuthof 180 degrees or more, and the second fisheye lens captures anotherhemispherical image having an image capturing azimuth of 180 degrees ormore.

The controller performs given processing for each of the hemisphericalimages, and combines or stitches the two hemispherical images togenerate a full-view spherical image, and transmits the full-viewspherical image to the three information processing apparatuses 100 a to100 c that request the full-view spherical image via the communicationinterface. The full-view spherical camera has the above describedconfiguration. Since the hardware configuration and functions of thefull-view spherical camera is known, the detailed hardware configurationand functions of the full-view spherical camera is not described.

Each of the three information processing apparatuses 100 a to 100 ccommunicates with the image capture device 110 wirelessly to acquire afull-view spherical image captured by the image capture device 110, anddisplays the full-view spherical image on a display used by each of thethree information processing apparatuses 100 a to 100 c. Each of thethree information processing apparatuses 100 a to 100 c can be anyapparatuses that can communicate with the image capture device 110wirelessly, can display the acquired full-view spherical image, and canperform various processing to the full-view spherical image. In anexample case illustrated in FIG. 1, the information processing apparatus100 a is, for example, a smart phone, the information processingapparatus 100 b is, for example, a tablet personal computer (PC), andthe information processing apparatus 100 c is, for example, a notebookpersonal computer (PC). The information processing apparatus 100 is notlimited to these, but can be a desktop PC, a digital camera, aprojector, an electronic information board, and a multifunctionperipheral (MFP).

In an example case illustrated in FIG. 1, the three informationprocessing apparatuses 100 a to 100 c and the image capture device 110are wirelessly connected by a wireless local area network (LAN) or thelike, but not limited to thereto. For example, the three informationprocessing apparatuses 100 a to 100 c and the image capture device 110can be connected by wire such as a cable or the like. Further, theconnection of the three information processing apparatuses 100 a to 100c and the image capture device 110 is not limited to the direct wirelessconnection or direct wired connection, but the three informationprocessing apparatuses 100 a to 100 c and the image capture device 110can be connected via a network such as the Internet.

The image capture system 1000 can be configured as above described.Hereinafter, a description is given of a hardware block diagram of theinformation processing apparatus 100 of the embodiment with reference toFIG. 2. Each of the three information processing apparatuses 100 a to100 c illustrated in FIG. 1 can employ the hardware block diagramillustrated in FIG. 2.

The information processing apparatus 100 includes, for example, aprocessor 10, a read-only memory (ROM) 12, a random access memory (RAM)13, an auxiliary storage 14, an input/output interface (I/F) 15, and acommunication interface (I/F) 16. The processor 10 controls operationsof the information processing apparatus 100 entirely.

The ROM 12 stores a boot program and a firmware program and the like.The RAM 13 provides an execution or working space of one or moreprograms. The auxiliary storage 14 stores one or more programs and anoperating system (OS) that are executed to implement each units insidethe information processing apparatus 100 to be described later. Theinput/output I/F 15 is used for connecting with a display 18. Thecommunication I/F 16 is used for establishing communication with theimage capture device 110.

The processor 10 reads the programs from the auxiliary storage 14 andexecutes the programs, and requests an acquisition of a full-viewspherical image to the image capture device 110 via the communicationI/F 16. Further, under the execution of the programs, the processor 10acquires the full-view spherical image via the communication I/F 16,saves or stores the acquired full-view spherical image in the RAM 13 orthe auxiliary storage 14, displays the acquired full-view sphericalimage on the display 18, and executes various processing to thefull-view spherical image. The details of these processing is describedlater.

First Embodiment

FIG. 3 illustrates an example of an appearance of the informationprocessing apparatus 100 of a first embodiment of the disclosure. Theinformation processing apparatus 100 includes the display 18, andinstalled with one or more programs used for executing variousprocessing such as one or more application programs (hereinafter,specific application) used for generating an animation sequentiallydisplaying two or more regions included in a full-view spherical image.In an example case illustrated in FIG. 3, the information processingapparatus 100 employs a smartphone, but not limited thereto. Forexample, the information processing apparatus 100 can be a tablet PC, anotebook PC, or a desktop PC. In the following description, theinformation processing apparatus 100 employs a smartphone.

As illustrated in FIG. 3, an operation screen of the specificapplication includes, for example, a section 20, a section 22, and asection 23 on the display 18, in which the operation screen may be alsoreferred to as an application screen or a screen. The section 20displays a two dimensional image that is generated by performing aprojective transformation to a partial region of a full-view sphericalimage, which corresponds to a three dimensional model. The section 22displays one or more thumbnail images of one or more registeredviewpoints to be described later. Further, the section 23 displaysvarious operation buttons such as a “slider,” a “preview” button, a“delete” button, a “save” button, and a “detail setting” button. Theslider is used for setting a transition speed to be described later. The“preview” button is used for playing generated movie image data as apreview to be described later. The “delete” button is used for deletingthe registered viewpoint to be described later. The “save” button isused to store the generated movie image data to be described later. Thedetail setting” button is used for setting detail settings to bedescribed later.

Hereinafter, a description is given of a functional configuration of theinformation processing apparatus 100 with reference to a functionalblock diagram illustrated in FIG. 4. In the information processingapparatus 100, when the processor 10 executes the above describedspecific application, functions of the information processing apparatus100 can be implemented, which means when the information processingapparatus 100 executes the specific application, the informationprocessing apparatus 100 can implement the respective functional units.In the following description, the information processing apparatus 100can implement the functional units when the specific application isexecuted, but not limited thereto. For example, a part or all of thefunctions of the information processing apparatus 100 can be implementedby a hardware such as a dedicated circuit. Further, a part or all of thefunctions of the information processing apparatus 100 can be implementedby a combination of software and hardware such as a dedicated circuit.

As illustrated in FIG. 4, the information processing apparatus 100includes, for example, a display unit 102, a control unit 103, acalculation unit 104, a viewpoint management unit 105, a viewpointcontrol unit 106, a movie image data generation unit 107, and a storageregion 108 as functional units. The storage region 108 can beimplemented, for example, by the RAM 13 and/or the auxiliary storage 14.

The display unit 102 generates an image to be displayed on the display18, and displays one or more user interfaces (UIs) on the display 18, inwhich the one or more UIs are operable by a user.

When a full-view spherical image is mapped to a surface of a givenvirtual three dimensional object (e.g. inner surface of sphere) as athree dimensional model, the viewpoint management unit 105 registers andmanages one or more viewpoints designated by a user with an orderdesignated by the user.

When a first viewpoint and a second viewpoint next to the firstviewpoint are specified or designated in the three dimensional model ofthe full-view spherical image, the viewpoint control unit 106 performs atransition of viewpoints from the first viewpoint to the secondviewpoint along a transition path interpolating between the firstviewpoint and the second viewpoint.

The calculation unit 104 executes various image processing and geometriccalculation, and generates one or more partial images, in which eachpartial image has the center that matches each of viewpoints transitingalong the transition path when the partial images are being displayed onthe display 18. In this disclosure, the “partial image” means a twodimensional image obtained by performing the projective transformationto a partial region setting one viewpoint in the three dimensional modelof the full-view spherical image as the center of the partial region.

The control unit 103 performs an animation display of a plurality ofgenerated partial images by connecting the plurality of generatedpartial images in an order of the transitions of the viewpoints.

The movie image data generation unit 107 converts the plurality ofgenerated partial images into movie image data using a general-purposefile format.

The storage region 108 stores various data.

The information processing apparatus 100 can be functionally configuredas above described. Hereinafter, a description is given of a use of thespecific application of the embodiment.

The display unit 102 of the information processing apparatus 100displays a partial image generated by the calculation unit 104 on thesection 20 of the display 18 (e.g., touch panel) of a smartphone, andthe display unit 102 receives a change of viewpoint and a change of anangle of view of a full-view spherical image such as “pan, tilt, zoomin, and zoom out” in response to a change operation performed by a useron the section 20. FIG. 5 including FIGS. 5(a) to 5(d) illustrateexamples of partial images displayed on the section 20 in accordancewith an operation of a user such as pan, tilt, zoom in, and zoom outoperation.

When the specific application is used, at first, as illustrated in FIG.6(a), a user performs an operation such as a long press or double tap todesignate or specify a desired position of a partial image displayed onthe section 20 as a first viewpoint (hereinafter, also referred to asviewpoint 1). In response to this operation on the section 20, theapplication screen transits to a state illustrated in FIG. 6(b), inwhich an icon 21 indicating the registration of the viewpoint 1 isdisplayed on the section 20 at a position within the partial imagedesignated by the user, and an icon 24 is displayed as a thumbnail imagefor calling the viewpoint 1.

Then, the user operates the section 20 to change the viewpoint todisplay another partial image, and then the user designates a desiredposition in the partial image as a second viewpoint (hereinafter, alsoreferred to as viewpoint 2) as illustrated in FIG. 7(a). In response tothe user operation on the section 20, the application screen transits toa state illustrated in FIG. 7(b), in which an icon 21 indicating theregistration of viewpoint 2 is displayed on the section 20 at theposition within the partial image designated by the user, and an icon 25is displayed as a thumbnail image for calling the viewpoint 2, in whichthe icon 25 is displayed next to the icon 24.

In the embodiment, when the number of registered viewpoints becomes twoor more (hereinafter referred to as the registered viewpoint orviewpoints), the two or more registered viewpoints are connected, andplayed as a preview of an animation.

In the above described example case, when the user selects the “preview”button displayed on the section 23 on the application screen illustratedin FIG. 7(b), an animation illustrated in FIG. 8 is played and displayedon the section 20. The user can check or confirm whether the animationof the full-view spherical image is generated in line with the userintention by using the preview playing function. Further, when thedisplay of the animation is completed up to the viewpoint 2, a playstart icon 30 is displayed (FIG. 8). By tapping the play start icon 30,the preview of the animation is executed again.

In the embodiment, the minimum unit of the viewpoint registration is theregistration of two viewpoints such as “viewpoint 1” and “viewpoint 2”respectively registered as a start viewpoint and an end viewpoint usedfor one transition. Then, the number of the registered viewpoints can beincreased by assuming the most-recently registered viewpoint (i.e., themost-recently registered end viewpoint) as a new start viewpoint, andregistering a new viewpoint as a new end viewpoint. By repeating thisprocedure, the number of the registered viewpoints can be increased.Further, the information processing apparatus 100 can be configured thatthe application registers a start point (i.e., viewpoint) of ananimation as a default point, in which the minimum unit of the viewpointregistration can be set when the user designates at least one viewpoint.

For example, as illustrated in FIG. 9, four viewpoints such as viewpoint3, viewpoint 4, viewpoint 5, and viewpoint 6 can be added to the abovementioned two viewpoints such as viewpoint 1 and viewpoint 2. When theviewpoint is to be registered, the angle of view can be changed byperforming a zoom-in or zoom-out operation to the partial imagedisplayed on the section 20. When the viewpoints having different angleof views are registered, a performance of zoom-in/zoom-out can beincorporated in the animation.

FIG. 10A(a) illustrates an example of an application screen when theabove described six viewpoints 1 to 6 are registered. In this examplecase, six icons corresponding to the six viewpoints are displayed as sixthumbnail images on the section 22.

When three or more viewpoints are registered as illustrated in FIG.10A(a), a viewpoint other than the registered first viewpoint and theregistered end viewpoint can be used as a start viewpoint so that apreview can be played from the viewpoint at an intermediate position ofthe three or more viewpoints (e.g., viewpoints 2, 3, 4, 5 in FIG.10A(a)), with which the check time of animation can be reduced.

Further, in the embodiment, each viewpoint can be called from the eachicon displayed on the section 22 as the thumbnail image, and thencontents of each viewpoint can be edited.

For example, when a user selects an icon of “viewpoint 4” as illustratedin FIG. 10A(b), one partial image currently displayed on the section 20is switched to another partial image setting the “viewpoint 4” as thecenter of another partial image. In response to this selectionoperation, the user can operate another partial image, switched from theone partial image, to edit the contents of the viewpoint 4 asillustrated FIGS. 10A(b) and 10B(c). FIGS. 10A(b) and 10B(c) illustratean example case that the coordinates of the registered viewpoint ischanged, but not limited thereto. For example, when the viewpoint iscalled by performing the above procedure, other parameter associatedwith the viewpoint can be edited.

Further, when a user selects one icon corresponding to one viewpoint,and then selects the “delete” button displayed on the section 23, theselected one viewpoint can be deleted. Further, when a user selects oneicon corresponding one viewpoint, the user can change a “transitionspeed,” to be described later, by using a slider displayed on thesection 23.

Further, in the embodiment, as illustrated in FIG. 11A(a), when a userselects the “detail setting” button displayed on the section 23, theapplication screen transits to a detail setting screen illustrated inFIG. 11B(b). As illustrated in FIG. 11B(b), the detail setting screenincludes, for example, a section 26, a section 27, and a section 28.

A slider for setting the transition speed, and a numerical input box forsetting the transition time are associated with each registeredviewpoint on the section 26, in which the transition speed and thetransition time are alternatively selected by using radio buttons. Withthis configuration, a user can adjust a transition pattern of theviewpoints by adjusting the transition speed and/or the transition time.

In this disclosure, the “transition speed” means a speed of transitionof the viewpoints from the first viewpoint to the second viewpoint alonga transition path interpolating between the first viewpoint and thesecond viewpoint when the first viewpoint and the second viewpoint, nextto the first viewpoint, are designated or specified in the threedimensional model of the full-view spherical image. The “transitiontime” means the time required for the transition from the firstviewpoint to the second viewpoint.

Further, a pull-down menu for selecting an easing curve is associatedwith each registered viewpoint on the section 26 as illustrated in FIG.11B(b). With this configuration, a user can change the transitionpattern of the viewpoints by using the easing curve. In this disclosure,the easing curve means a curve indicating a speed change of an animationover the time. When “linear” is set, the transition of viewpoints occursat a constant speed, when “easeIn” is set, the transition of viewpointsoccurs at a slow speed at first, gradually accelerated, and then at aconstant speed, when “easeOut” is set, the transition of viewpointsoccurs at a fast speed at first, gradually decelerated, and then at aconstant speed, and when “easeInOut” is set, performances of “easeIn”and “easeOut” are combined.

In this disclosure, each of the above described parameters may bereferred to as a “viewpoint parameter.” In the embodiment, eachviewpoint parameter can be selectively set to each viewpoint, with whichan animation can be displayed with various patterns such as a dynamicpattern having various changing pattern in the animation playing.

Further, the section 27 displays an icon for selecting “entire setting.”The “entire setting” icon is associated with a slider for setting thetransition speed, the numerical input box for setting the transitiontime, and the pull-down menu for selecting the easing curve. When the“entire setting” icon is selected, the above described each of theparameters is applied to the entire viewpoints such as the firstviewpoint to the end viewpoint (i.e., last viewpoint), with whichfeeling of unity can be set for an animation, and natural movement canbe performed for the animation. For example, the “entire setting” issuitable when to display the animation while continuously rotating inthe same direction.

Further, the section 28 displays icons for selecting a preset of theviewpoint parameter such as preset 1 to preset 5. In this disclosure,one or more sets of viewpoint parameters and representative transitionpattern (e.g., transition path) of viewpoints can be prepared as presetdata, and the preset data can be stored in the storage region 108 inadvance. When a user selects an icon displayed on the section 28, presetdata corresponding to the selected icon is read out from the storageregion 108, and the preset data is applied to a full-view sphericalimage read out from a memory by a user operation, with which theanimation display is performed automatically. Also in this case, theicon used for calling the viewpoint is displayed on the section 22 asthe thumbnail image. In this example case, the user can use theanimation that is automatically performed in accordance with the presetdata, or the user can customize the animation by editing any of theviewpoints displayed on the section 22. Further, in another embodiment,a set of viewpoint parameters corresponding to the animation displayprepared by the user can be stored in the storage region 108 as presetdata so that the viewpoint parameters can be used again.

Further, in the embodiment, when a user selects the “save” buttondisplayed on the section 23, as illustrated in FIG. 12A(a), theapplication screen transits to a state illustrated in FIG. 12A(b).During this period, the animation generated by applying the displaymethod using the above described procedure is cropped with a size of thescreen, converted into movie image data using a general-purpose fileformat such as Moving Picture Experts Group (MPEG) or Audio VideoInterleaved (AVI), and then saved or stored in the storage region 108.Further, in another embodiment, the animation can be cropped at any cropregion other than the size of the screen, or any background audio can besynthesized with the animation.

When the animation is converted into the movie image data using thegeneral-purpose file format as above described, the movie image data canbe played simply by using a standard movie image play applicationwithout using a dedicated viewer of the full-view spherical image. FIG.12B(c) illustrates an example case that the movie image data is playedby using the standard movie image play application. Therefore, in theembodiment, the partial images having different viewpoints are generatedfrom the full-view spherical image, converted to the movie image datausing the general-purpose image file format, and the converted movieimage data is played while transiting the viewpoints from one viewpointto the next viewpoint. The movie image data using the general-purposeimage file format can be posted on a movie image posting site, or can beshared with others on social networking service (SNS).

The specific application of the embodiment can be used as abovedescribed. Hereinafter, a description is given of processing executed byeach unit of the information processing apparatus 100 (see FIG. 4).Hereinafter, a description is given of processing of the informationprocessing apparatus 100 that is executed when registering the viewpointwith reference to a sequence diagram illustrated in FIG. 13.

At first, a user performs a registration operation of a viewpoint (S1)by performing the above described procedure. In response to theviewpoint registration operation at S1, the display unit 102 reads andacquires the following viewpoint parameters (1) and (2) (S2).

-   -   (1) currently set transition speed or transition time    -   (2) currently set easing curve

Then, the display unit 102 reports the control unit 103 that theviewpoint registration is requested from the user by using theparameters acquired at S2 as argument (S3).

In response to this report, the control unit 103 instructs thecalculation unit 104 to calculate spherical coordinates (θ, φ) of theviewpoint corresponding to image coordinates (x, y) of the viewpointdesignated or specified by the user, and acquires a calculation resultfrom the calculation unit 104 (S4). Then, the control unit 103 instructsthe calculation unit 104 to calculate a current display magnificationsuch as zoom magnification with respect to an initial angle of view, andacquires a calculation result from the calculation unit 104 (S5).Hereinafter, the spherical coordinates (θ, φ) of the viewpoint is alsoreferred to as the viewpoint coordinates.

Then, the control unit 103 reports the parameters (1) and (2) acquiredfrom the display unit 102, and the spherical coordinates (θ, φ) of theviewpoint and the display magnification acquired from the calculationunit 104 to the viewpoint management unit 105, and requests aregistration of a new viewpoint (S6).

In response to this request, the viewpoint management unit 105 newlygenerates a viewpoint information management table 500 illustrated inFIG. 14A, and registers each value reported from the control unit 103 inthe viewpoint information management table 500. As illustrated in FIG.14A, the viewpoint information management table 500 includes fields 501to 507. The field 501 stores an index of the registered viewpoint. Thefield 502 stores a horizontal angle (θ) of the registered viewpoint. Thefield 503 stores an elevation angle (φ) of the registered viewpoint. Thefield 504 stores the display magnification. The field 505 stores thetransition speed. The field 506 stores the transition time. The field507 stores the easing curve. The viewpoint management unit 105 storeseach value (hereinafter, referred to as viewpoint information) reportedfrom the control unit 103 at S6 in the respective fields in theviewpoint information management table 500.

In response to the completion of registration of the viewpoint, thecontrol unit 103 returns an icon of a thumbnail image corresponding tothe registered viewpoint to the display unit 102 (S7).

Then, the display unit 102 displays the icon of thumbnail image on thesection 22 (S8) to inform the user that the viewpoint registration iscompleted.

Each time the user performs the above described viewpoint registrationprocessing, S1 to S8 are repeatedly executed, and the viewpointdesignated or specified by the user is registered in the viewpointinformation management table 500 with the order designated or specifiedby the use.

FIG. 14B illustrates an example of the viewpoint information managementtable 500 after six viewpoints is registered. In this processing, themost-recently registered viewpoint (e.g., viewpoint 6) becomes themost-recently registered end viewpoint, and the fields 505 to 507corresponding to the most-recently registered end viewpoint stores“NONE” to indicate that there is no value in the fields 505 to 507. Inthis disclosure, the viewpoint information management table 500 isstored in the storage region 108 by associating the viewpointinformation management table 500 as meta data of the full-view sphericalimage.

Further, in response to the above described editing operation of theviewpoint, the values of respective fields in the viewpoint informationmanagement table 500 are updated. Further, in addition to using theviewpoint information management table 500 as the meta data of thefull-view spherical image, the contents of the viewpoint informationmanagement table 500 alone can be registered as preset data. Further,the viewpoint information management table 500 can be output to anexternal database by associating with the full-view spherical image, orthe viewpoint information management table 500 alone can be output tothe external database.

The information processing apparatus 100 performs the viewpointregistration as described above. Hereinafter, a description is given ofprocessing of the information processing apparatus 100 when executingthe preview playing with reference to a sequence diagram illustrated inFIG. 15.

At first, when a user operates the preview playing based on the abovedescribed procedure (S1A), in response to this operation, the displayunit 102 reports the control unit 103 that the playing of animation isrequested from the user (S2A).

In response to this report, the control unit 103 requests the viewpointinformation associated with a full-view spherical image to the viewpointmanagement unit 105, and acquires a set of viewpoint informationassociated with the full-view spherical image, stored in the viewpointinformation management table 500 stored in the storage region 108, fromthe viewpoint management unit 105 (S3A).

Then, the control unit 103 requests the start of controlling theanimation to the viewpoint control unit 106 by using the set ofviewpoint information of the registered first viewpoint (i.e., startviewpoint) and the registered second viewpoint (i.e., end viewpoint) asthe argument (S4A).

In response to this request, the viewpoint control unit 106 performs acontrol of transition of the viewpoints from the registered firstviewpoint (i.e., start viewpoint) to the registered second viewpoint(i.e., end viewpoint) along a transition path interpolated between theregistered first viewpoint (i.e., start viewpoint) and the registeredsecond viewpoint (i.e., end viewpoint).

Specifically, the viewpoint control unit 106 calculates a shift value ofthe viewpoint for each frame (i.e., each partial image configuring oneanimation) based on the distance between the registered two viewpointssuch as the registered first viewpoint (i.e., start viewpoint) and theregistered second viewpoint (i.e., end viewpoint), the transition speedor the transition time set for transiting from the registered firstviewpoint (i.e., start viewpoint) to the registered second viewpoint(i.e., end viewpoint), in which the transition speed and the transitiontime are the viewpoint parameters set for the registered first viewpoint(SSA). At S5A, the viewpoint control unit 106 calculates the shift valueof the viewpoint under an assumption that the viewpoints transit fromthe registered first viewpoint (i.e., start viewpoint) to the registeredsecond viewpoint (i.e., end viewpoint) by interpolating the shortestdistance between the registered first viewpoint (i.e., start viewpoint)and the registered second viewpoint (i.e., end viewpoint).

After completing the calculation of the shift value of the viewpoint foreach frame, the viewpoint control unit 106 reports the start of theanimation to the control unit 103 (S6A).

Then, the viewpoint control unit 106 repeatedly reports the shift valueof the viewpoint for each frame to the control unit 103 until a valueobtained by multiplying a frame period to the number of frames exceedsthe transition time (S7A).

When the control unit 103 receives the report of the shift value of theviewpoint from the viewpoint control unit 106, the control unit 103requests the calculation unit 104 to calculate a shift of viewpoint, andacquires a partial image setting the shift-calculated viewpoint as thecenter of the partial image as a calculation result (S9A).

In response to this acquiring, the control unit 103 transmits thepartial image acquired at S9A to the display unit 102, and requestsdisplay unit 102 to update the displaying of the partial image (S10A).

By contrast, when the value obtained by multiplying the frame period tothe number of frames exceeds the transition time, the viewpoint controlunit 106 reports to the control unit 103 that the animation startingfrom the registered first viewpoint (i.e., start viewpoint) to theregistered second viewpoint (i.e., end viewpoint) is completed (S8A),and the sequence proceeds to the next processing such aspost-processing. The processing from S4A to S10A is repeatedly executedfor the number of registered viewpoints configuring one animation thatis from the start viewpoint to the end viewpoint, and when the number ofanimation count has reached the total number of the registeredviewpoints, the control unit 103 reports to the display unit 102 thatthe animation is completed (S11 A).

By performing the above described procedure, the control unit 103executes the animation of a plurality of the partial images generated bythe calculation unit 104 and connected with an order of the transitionof the viewpoints.

In this configuration, in addition to the interpolation between theviewpoints, it is preferable that the angle of view is interpolatedbetween the registered two viewpoints when performing the transition ofviewpoints. In this case, the viewpoint control unit 106 calculates ashift value of the angle of view for each frame based on a difference ofthe display magnification between the registered two viewpoints such asthe adjacent viewpoints, and the calculation unit 104 generates one ormore partial images having the angle of view changed for each framebased on the calculated shift value of the angle of view. With thisconfiguration, a smooth zoom-in and/or zoom-out expression can beperformed when playing the animation.

Further, the movie image data generation unit 107 converts the pluralityof partial images generated by the calculation unit 104 into movie imagedata using a general-purpose file format such as Moving Picture ExpertsGroup (MPEG) or Audio Video Interleaved (AVI), and stores the movieimage data in the storage region 108.

As described above, in this disclosure, a concerned portion or region inthe full-view spherical image can be easily presented as a movie imagewithout causing longer time and greater load for searching and selectingthe viewpoints by a user.

The first embodiment has the above described features. Hereinafter, adescription is given of a second embodiment of the disclosure. In thefollowing description, the description of portions in common with thefirst embodiment are omitted, and differences from the first embodimentare described.

Second Embodiment

In the above described first embodiment, it is assumed that theviewpoints shift from the registered first viewpoint (i.e., startviewpoint) to the registered second viewpoint (i.e., end viewpoint) byinterpolating the shortest distance between the registered firstviewpoint (i.e., start viewpoint) and the registered second viewpoint(i.e., end viewpoint), and the viewpoint control unit 106 calculates theshift value of viewpoint. In the second embodiment, a user can set aviewpoint transition direction as a viewpoint parameter, which isdifferent from the first embodiment.

In the second embodiment, as illustrated in FIG. 16A(a), when a userdesignates a viewpoint in a partial image displayed on the section 20,an application screen transits to a state illustrated in FIG. 16A(b),and an icon 29 indicating the registration completion of the viewpointis displayed in the partial image.

The icon 29 includes a direction instruction button having four arrowsfor specifying the vertical direction (i.e., tilt direction) and thelateral direction (i.e., pan direction) of the partial image. Asillustrated in FIG. 16B(c), in response to a selection of any one of thefour arrows of the direction instruction button by the user, atransition direction to the next to-be-registered viewpoint is set. Inan example case of FIG. 16, the direction instruction button has fourarrows for specifying or instructing four directions such as up, down,left, and right, but the direction instruction button can be set witheight directions to set the transition direction more finely. Further,when the second and succeeding viewpoint are to be registered, thedirection instruction button can be configured to display an arrowforwarding or directing to the second and succeeding viewpoint to beregistered, in which the transition direction from the most-recentlyregistered viewpoint toward the next to-be-registered viewpoint can bespecified.

FIGS. 17A to 17D illustrate examples of relationships between aselection of the direction instruction button in the icon 29 and atransition direction from the registered first viewpoint to theregistered second viewpoint. In the second embodiment, the transitiondirection is designated when the viewpoints are registered. Therefore,as illustrated in FIGS. 17A to 17D, a transition path of the viewpointscan be set with various patterns such as the shortest route and thedetour, with which the animation can be performed with various patterns.

In the second embodiment, when the transition direction is set whileregistering the viewpoint, the transition direction is registered in theviewpoint information management table as one of the viewpointparameters. FIG. 14C illustrates a viewpoint information managementtable 550 generated in the second embodiment. As illustrated in FIG.14C, the viewpoint information management table 550 further includes afield 508 that stores the transition direction in addition to the abovedescribed fields 501 to 507. The second embodiment is described asabove. Hereinafter, a description is given of a third embodiment of thedisclosure. In the following description, the description of portions incommon with the previous embodiments are omitted, and only thedifferences may be described.

Third Embodiment

In the above described embodiments, (1) automatic execution of animationusing preset data, (2) crop of animation, and (3) synthesis of animationand background audio are schematically described. In the thirdembodiment, (1) to (3) will be described in detail with reference toother example cases.

FIG. 18(a) illustrates an example of a preset data 600 of the thirdembodiment. The preset data 600 registers two or more viewpoints in afull-view spherical image with a given order, and the preset data 600 isused as data for generating an animation, in which the preset data 600is used similar to the viewpoint information management table 500 (seeFIG. 14). In the third embodiment, it is assumed that one or more of thepreset data 600 are stored in the storage region 108.

As illustrated in FIG. 18(a), the preset data 600 includes, for example,fields 601 to 604. The field 601 stores an index of the registeredviewpoint. The field 602 stores a horizontal angle (θ) of the registeredviewpoint. The field 603 stores an elevation angle (y) of the registeredviewpoint. The field 604 stores display magnification such as a zoommagnification with respect to the initial angle of view. In the thirdembodiment, when the preset data 600 is used, the same “transitionspeed” and “easing curve” are applied for the entire viewpoints similarto the above described “entire setting.” Therefore, the “transitionspeed” and “easing curve” are not registered in the preset data 600.

Further, in the third embodiment, it is assumed that one or more cropregions are stored in the storage region 108. The crop region is arectangular frame used for defining a region where the animation imageis cropped, and in the third embodiment, a rectangular frame having agiven aspect ratio can be registered as the crop region.

Further, in the third embodiment, it is assumed that one or morebackground audio are stored in the storage region 108. The backgroundaudio means any type of audio data, and in the third embodiment, audiodata desired for a user can be registered as the background audio.

FIG. 19(a) illustrates an example of an application screen in the thirdembodiment. As illustrated in FIG. 19(a), in the third embodiment, thesection 23 of the application screen displays a “preset” button forcalling the preset data. When a user selects the “preset” buttondisplayed on the section 23, the application screen transits to apreset-use screen illustrated in FIG. 19(b).

As illustrated in FIG. 19(b), the preset-use screen includes, forexample, a section 40, a section 42, a section 43, and a section 44. Inthe third embodiment, a partial image of a full-view spherical image isdisplayed on the section 40. Further, the section 43 displays, forexample, a “preset” button for calling the preset data 600, a “sound”button for calling background audio, a “crop” button for calling a cropregion, and a “transition speed” button for setting a transition speed.Further, the section 44 displays, for example, a “preview” button forrequesting preview playing, and a “save” button for storing thegenerated movie image data. Further, the section 42 displays one or moreUIs corresponding to each button displayed on the section 43.

For example, in the third embodiment, when a user selects the “preset”button displayed on the section 23 (FIG. 19(a)), as illustrated in FIG.20A(a), one or more icons corresponding to the preset data 600 stored inthe storage region 108 are displayed on the section 42 as selectableoptions or selections. When the user selects any of the icons inresponse to displaying of the one or more icons on the section 42, thepreset data 600 corresponding to the selected icon is read out from thestorage region 108, and set in a temporal storage such as a memory. Inan example case illustrated in FIG. 20A(a), each icon displays aschematic image of contents set in the preset data 600.

Further, when the user selects the “sound” button displayed on thesection 43, as illustrated in FIG. 20A(b), one or more iconscorresponding to the background audio stored in the storage region 108are displayed on the section 42 as selectable options or selections.When the user selects any of the icons displayed on the section 42,audio data corresponding to the selected icon is read out from thestorage region 108, and set as the background audio. Further, when theuser does not select the background audio, a default value such assilence or sound mute is set.

Further, when the user selects the “transition speed” button displayedon the section 43, as illustrated in FIG. 20B(c), a slider for settingthe transition speed is displayed on the section 42. When the useroperates the slider of the transition speed to designate the transitionspeed, the designated transition speed is set as the transition speed ofthe viewpoints when generating an animation. Further, when the user doesnot designate the transition speed, a default value or a giventransition speed is set. Further, the easing curve is set with, forexample, a default value (e.g. linear).

Further, when the user selects the “crop” button displayed on thesection 43, as illustrated in FIG. 21A(a), one or more iconscorresponding to the crop regions stored in the storage region 108 aredisplayed on the section 42 as selectable options or selections. Whenthe user selects any of the icons in response to displaying of the oneor more icons on the section 42, a rectangular frame patterncorresponding to the selected icon is read out from the storage region108, and set as the crop region.

For example, when the user selects “2:1” as the crop region, theanimation is cropped as illustrated in FIG. 21A(b). Further, when theuser selects “free” as the crop region, as illustrated in FIG. 21B(c), arectangular frame 45 is displayed on the section 40. In this case, asize of the frame 45 can be variably changed, in which the crop regioncan be freely set by a user by manually changing the size of the frame45. Further, when the user does not select the crop region, a defaultvalue (e.g., size of screen) is set.

In the third embodiment, the user can variously change and set the abovedescribed each value (e.g., preset data, background audio, transitionspeed, crop region), and the preview can be played each time the valueis changed and set.

For example, as illustrated in FIG. 22A(a), when a user selects an iconQ displayed on the section 42, and then selects the “preview” button,the control unit 103 acquires the preset data 600 corresponding to theicon Q from the storage region 108 via the viewpoint management unit105. Then, based on a set of the viewpoints and the displaymagnification (angle of view) stored in the acquired preset data 600,the control unit 103, the viewpoint control unit 106, the calculationunit 104, and the display unit 102 cooperate to execute the previewplaying to display the animation. The contents of processing to beexecuted at the time of preview playing is stored in the preset data 600instead of the above described viewpoint information management table500. Since the viewpoint information management table 500 described withreference to FIG. 15 and the preset data 600 are substantially the same,a detailed description is omitted.

FIG. 22A(b) illustrates an example of an application screen when thepreview playing is completed. When the preview playing is completed, anend frame image of the animation is displayed with a stopped state onthe section 40. When the animation is uploaded to a video sharing siteor SNS, viewers see the end frame image of the animation, and evaluatethe animation based on the end frame image, in which impression of theend frame image of the animation may determine the evaluation of theanimation by the viewers. Further, it is not rare that the end frameimage is used as a thumbnail image of the animation uploaded to an opensite on the network.

When the preset data 600 is used, the end frame image of the animationis determined by one viewpoint and the angle of view of the oneviewpoint registered as the end viewpoint in the preset data 600.However, the contents of the end frame image corresponding to theregistered end viewpoint may be determined without consideration ofimage impression, in which the contents of the end frame image may notexpress impression matched to an intention of a person or an entity thatcreates the animation. For example, when an animation as a whole is avery interesting one, but the end frame image happens to be an imagehaving poor characteristics, there is a possibility that viewers feelnegative impression, and further, if the image having poorcharacteristics is used as a thumbnail image, viewer's attention may notbe attracted.

In view of this issue, in the third embodiment, as illustrated in FIG.22A(b), an end frame adjustment button 50 is displayed on the section 40when the preview playing is ended. When a user selects the end frameadjustment button 50, the user can change the end frame image of theanimation to the end frame image matched to contents desired for a user.

When the user selects the end frame adjustment button 50, the displayunit 102 transits a state of the section 40 to a state that can receivea screen operation by the user (e.g., pan, tilt, zoom in, zoom out) toreceive a change operation of the end viewpoint and the angle of viewassociated with the end viewpoint registered in the preset data 600.

As illustrated in FIG. 22B(c), when the user designates a new viewpointand a new angle of view by performing an operation on the section 40 inresponse to the selection of the end frame adjustment button 50, thecontrol unit 103 replaces or updates the end viewpoint and the angle ofview associated with the end viewpoint registered in the preset data600, which is currently stored in the temporal storage, to the newviewpoint and the new angle of view newly designated by the user.

FIG. 18(b) illustrates an example case that a value of coordinates and adisplay magnification (i.e., angle of view) of the viewpoint 6registered in the preset data 600 as the end viewpoint are replaced orupdated. At this timing, an icon 21 indicating the registrationcompletion of a new viewpoint is displayed on the section 40 asillustrated in FIG. 22B(c). The processing that is performed when theuser selects the end frame adjustment button 50 is substantially thesame processing that is executed when selecting the icon of theviewpoint and then editing the contents of the viewpoint in the previousembodiment (see FIG. 10), and thereby a detailed description thereof isomitted.

When the user selects the “preview” button at this timing, the animationis generated based on the updated preset data 600. In this case, theviewpoint control unit 106 transits the viewpoints along a modifiedtransition path interpolated based on the changed end viewpoint, thecalculation unit 104 generates a plurality of partial images based onthe viewpoints transiting along the modified transition path and thechanged angle of view of the end viewpoint, and the control unit 103connects the plurality of partial images generated in this way as ananimation, and display the animation on the section 40. At this timing,the control unit 103 crops the animation based on the crop region thatis currently set. If the background audio has been set currently, thebackground audio is played by synchronized with the playing ofanimation.

Further, when the user selects the “save” button at this timing, themovie image data generation unit 107 converts the plurality of partialimages, generated by a procedure similar to the above describedprocedure, to movie image data using a general-purpose motion picturefile format, and stores the movie image data in the storage region 108.At this timing, the movie image data generation unit 107 crops theanimation based on the crop region that is currently set, and if thebackground audio has been set currently, the movie image data generationunit 107 synthesizes the background audio with the movie image data.

As to the above described third embodiment, the movie image data thatpresents a concerned portion of a full-view spherical image can begenerated by using the preset data prepared in advance. As to the thirdembodiment, even when the preset data is used, the end frame image ofthe movie image data can be matched to a user interest or intension.

The third embodiment is described as above. Hereinafter, a descriptionis given of a fourth embodiment of the disclosure. In the followingdescription, the description of portions in common with the previousembodiments are omitted, and only differences is described.

Fourth Embodiment

In the above described first to the third embodiments, images can bedisplayed with various patterns by changing the parameters such as theviewpoint and the angle of view of the viewpoint in the full-viewspherical image. However, the parameters are not limited to thereto, butother parameters can be adjusted. For example, when a projection method,type or mode is changed between the viewpoints, an image can bedisplayed dynamically.

The projection type is a method of expressing a full-view sphericalimage mapped on a surface of a virtual three dimensional object (e.g.inner face of sphere) onto a two dimensional plane, and is similar to amap projection method. Before describing the projection type, theparameters for changing the image expression is described.

A full-view spherical image is mapped on the surface of a virtual threedimensional object. FIG. 23 illustrates a sphere 200 expressed on a twodimensional plane represented by the Y axis and the Z axis, which aretwo of the three axes of a three dimensional space represented by thethree axes of the X axis, the Y axis, and the Z axis. The Y axis is usedas the height direction (vertical direction), and the Z axis is used asthe horizontal direction parallel to a ground. In an example case ofFIG. 23, the center of the sphere 200 is set as the origin 201, and acamera 202 is disposed outside the sphere 200 on the Z-axis passingthrough the origin 201 to capture and acquire a scene for generating thepartial image. It should be noted that the sphere 200 is a virtualsphere, and the camera 202 is a virtual image capture, which areimplemented in the computing process in the information processingapparatus 100.

As described in the first to third embodiments, the parameters include,for example, a viewpoint specified as a desired position in a partialimage, and an angle of view “a” indicating a range of angle capturableby the camera 202.

As illustrated in FIG. 23, the parameters further include cameracoordinates indicating a position of the camera 202, camera UPindicating the upward direction of the camera 202 as a vector, cameragazing point coordinates indicating a point that the camera 202 isgazing (i.e., gazing point), and a radius of the sphere 200 that is avirtual three dimensional object used for mapping a full-view sphericalimage. Hereinafter, the camera gazing point may be also referred to asthe gazing point.

These parameters are just examples, and other parameters can be set. Itshould be noted that these parameters are parameters necessary forchanging the projection type or projection mode.

In an example case illustrated in FIG. 23, the radius of the sphere 200is set one (1), the gazing point is set at the origin 201 of the sphere200, the angle of view “a” is set with 120 degrees, the cameracoordinates is set as (0, 0, −1.8), and the camera UP is set as (0, 1,0).

Hereinafter, a description is given of the projection types or modeswith reference to FIGS. 24 to 26. It should be noted that the projectiontypes or modes illustrated in FIGS. 24 to 26 are just examples, and theprojection types or modes are not limited thereto. Hereinafter, similarto an example illustrated in FIG. 23, it is assumed that a full-viewspherical image is mapped onto the surface of the sphere 200, in whichthe sphere 200 has a radius of one (1), the sphere 200 has the center atthe origin 201, and the sphere 200 is expressed on the two dimensionalplane represented by the Y axis and the Z axis as illustrated in FIGS.24 to 26.

FIG. 24 illustrates an example of a first projection type or modeviewing images by disposing the camera 202 outside the sphere 200. Whenthe first projection type is applied, an image such as a wide rangeimage (e.g., panoramic image) mapped on a surface of a mirror ball canbe displayed.

When the first projection type is applied, as illustrated in FIG. 24A,the gazing point is set at the origin 201 that is the center of thesphere 200, and the camera 202 is disposed at a position outside theradius of the sphere 200. For example, the camera 202 is disposed at aposition distanced from the origin 201 for “1.8” on the Z axis outsidethe sphere 200, and the angle of view “α” is set with, for example, 120degrees. Therefore, as illustrated in FIG. 24B, the parameters includethe angle of view “α” of 120 degrees, the gazing point coordinates of(0, 0, 0), the camera coordinates of (0, 0, −1.8), and the camera UP of(0, 1, 0).

FIG. 25 illustrates an example of a second projection type or modeviewing images in the entire inside of the sphere 200 from the upperside by setting Y coordinate of the camera 202 equal to the radius ofthe sphere 200. When the second projection type is applied, an imagesuch as a person, a building or the like standing on a smaller planetcan be displayed.

When the second projection type is applied, as illustrated in FIG. 25A,the camera 202 is disposed inside the sphere 200 close to the boundaryof the sphere 200 as much as possible, the angle of view “α” is set witha ultra-wide angle of 170 degrees, and the gazing point is set at theorigin 201. Therefore, as illustrated in FIG. 25B, the parametersinclude, for example, the angle of view “α” of 170 degrees, the gazingpoint coordinates of (0, 0, 0), the camera coordinates of (0, 1, 0), andcamera UP of (0, 0, 1).

FIG. 26 illustrates an example of a third projection type or modeviewing images on the sphere 200 from the origin 201 by disposing thecamera 202 at the origin 201. When the third projection type is applied,an image radially extending from the center toward the periphery can bedisplayed.

When the third projection type is applied, as illustrated in FIG. 26A,the camera 202 is disposed at the origin 201, and an intersection pointof the circle representing the surface of the sphere 200 and the Z axisis set as the gazing point. Therefore, as illustrated in FIG. 26B, theparameters include, for example, the angle of view “α” of 205 degrees,the gazing point coordinates of (0, 0, 1), the camera coordinates of (0,0, 0), and the camera UP of (0, 1, 0).

As to the first to the third embodiments, one of the projection typessuch as the first projection type is used to register the viewpoints(e.g., viewpoint 1, viewpoint 2, . . . , viewpoint N) and connect theadjacent two registered viewpoints to execute the preview playing of theanimation. Therefore, the projection type is set same for all of theviewpoints. Further, since the gazing point is set at the origin 201,the gazing point is not considered as the parameter.

By contrast, in the fourth embodiment, each viewpoint and each gazingpoint are registered using different projection types such as the firstprojection type for the viewpoint 1, the second projection type for theviewpoint 2, in which the preview playing is performed by changing theprojection types while transiting the viewpoints and the gazing points.

Hereinafter, a description is given of a method of selecting theprojection type with reference to FIG. 27. When a user selects ato-be-displayed full-view spherical image file, and displays a partialimage using one projection type. the user selects a “type selection”button displayed on the section 23 of the display 18 (e.g., touch panel)of a smart phone as illustrated in FIG. 27(a) before performingoperations such as pan, tilt, zoom in, and zoom out operations to thedisplayed partial image on the section 20, as needed, to switch a screenillustrated in FIG. 27(a) to a screen used for selecting the projectiontype illustrated in FIG. 27(b).

The user can select one button from the “first projection type” button,the “second projection type” button, and the “third projection type”button displayed on the section 46 of the application screen asillustrated in FIG. 27(b). In this configuration, one of the projectiontypes can be set as a default projection type. When the user wants toperform a registration of the viewpoint using a projection type otherthan the default projection type, the user can select a desiredprojection type. In an example case illustrated in FIG. 27(b), the“second projection type” button is selected by the user. After selectingthe projection type, one partial image is switched to another partialimage applied with the selected projection type, in which a returnbutton 51 used for returning to the previous partial image is displayedat a upper side of the section 20 as illustrated in FIG. 27(b). When thereturn button 51 is pressed, another partial image illustrated in FIG.27(b) is returned to the previous partial image illustrated in FIG.27(a).

In the information processing apparatus 100 of the fourth embodiment,the display unit 102 receives a selection of the projection type inresponse to an operation of the user. The display unit 102 updates onepartial image, which is currently being displayed, to another partialimage applied with the selected projection type. When the projectiontype is not selected and the projection type is not changed, thisprocessing is not executed.

Specifically, the display unit 102 receives the selection of theprojection type, acquires one or more parameters associated with theselected projection type from the parameters set as the projectioninformation as illustrated in FIGS. 24B, 25B, and 26B, generates apartial image applied with the selected projection type using theacquired one or more parameters, and switches the partial image, whichare currently being displayed, to the generated partial image.

Then, similar to the first to third embodiments, the user performs agiven operation such as a long press or double tap to designate adesired position in the partial image displayed on the section 20 as theviewpoint 1. When the user designates the viewpoint 1, the user canchange the display magnification by using the zoom-in or zoom-outoperation. In response to this operation, the application screendisplays an icon indicating that the viewpoint 1 is registered at theposition designated by the user within the partial image, and displaysan icon of a thumbnail image used for calling the viewpoint 1 on thesection 22.

Then, the user can select the “type selection” button displayed on thesection 23 (FIG. 27(a)) to switch the application screen, and thenselects one button among the “first projection type” button, the “secondprojection type” button, and the “third projection type” buttondisplayed on the section 46 (FIG. 27(b)). The display unit 102 switchesthe currently-being-displayed partial image to the partial image appliedwith the selected projection type to update the partial image. In thiscase too, when the projection type is not selected, and the projectiontype is not changed, this processing is not executed.

Then, the use designates a desired position in the displayed partialimage as the viewpoint 2. When the user designates the viewpoint 2, theuser can change the display magnification by using the zoom-in orzoom-out operation. In response to this operation, the applicationscreen displays an icon indicating that the viewpoint 2 is registered atthe position designated by the user within the partial image, anddisplays an icon of a thumbnail image used for calling the viewpoint 2on the section 22.

By repeating the above described operation, viewpoint 3, viewpoint 4, .. . , and viewpoint N can be added in succession to the viewpoint 1 andthe viewpoint 2. Also in this case, when the number of the registeredviewpoint becomes two or more, a preview of the animation connecting thetwo or more registered viewpoints can be played.

Hereinafter, a description is given of processing that is performed whena viewpoint is registered by the information processing apparatus 100with reference to a sequence diagram illustrated in FIG. 28. At first, auser selects a projection type prior to the viewpoint registrationoperation (S1B). In response to this projection type selectionoperation, the display unit 102 acquires one or more parametersassociated with the selected projection type.

The parameter means one or more values set for each projection type suchas the camera coordinates used for displaying a partial image for eachprojection type, and managed by the viewpoint management unit 105 byusing a parameter table.

Specifically, the parameter table can be set as illustrated in FIGS.24B, 25B, and 26B. Therefore, the display unit 102 reports the selectedprojection type to the viewpoint management unit 105, acquires one ormore parameters associated with the selected projection type from theviewpoint management unit 105, in which the parameter means an initialvalue (i.e., default value) of the camera coordinates or the like set inadvance.

When the selected projection type is the first projection type, and theparameters such as the camera coordinates, the angle of view “α”, thegazing point coordinates, the camera UP are acquired, the display unit102 maps a full-view spherical image on a surface of the sphere 200having the radius, sets the camera 202 at the camera coordinates, setsthe direction of the camera UP to the upward direction of the camera202, generates a partial image setting a focus on the gazing pointcoordinates using the acquired angle of view “α”, and switches a partialimage, which is currently being displayed, to a generated partial imageto update the partial image.

Since the processing from the viewpoint registration operation by theuser (S3B) to the calculation of the current display magnification (S7B)is same as the processing of S2 to S5 illustrated in FIG. 13, adescription of the processing from S3B to S7B is omitted.

When the viewpoint registration operation is performed, the viewpoint ischanged, and the display magnification is changed, and thereby theviewpoint coordinates is calculated at S6B, and the displaymagnification is calculated at S7B. Further, when the viewpointregistration operation is performed, the upward direction of the camera202 and the camera coordinates of the camera 202 are changed by changingthe viewpoints and others, and thereby the upward direction of thecamera 202 and the camera coordinates of the camera 202 are alsorequired to be calculated.

Therefore, the control unit 103 instructs the calculation unit 104 tocalculate the camera UP, and acquires a calculation result of the cameraUP (S8B). The camera UP can be calculated from information of the upwarddirection of the full-view spherical image when designating theviewpoint, the viewpoint coordinates (i.e., spherical coordinates ofviewpoint) calculated at step S6B, and the gazing point coordinates.

When the first projection type and the second projection type areapplied, the gazing point coordinates is set at the coordinates of (0,0, 0) that is the origin 201 of the sphere 200, and thereby the gazingpoint coordinates is not required to be calculated. By contrast, whenthe third projection type is applied, the gazing point coordinates iscalculated as an intersection point coordinates of the circlerepresenting the surface of the sphere 200 and the straight lineindicating a camera gazing direction extending from the camera 202 setat the origin 201. The camera UP indicates a direction perpendicular toa straight line extending in a viewing direction of the camera 202, andis calculated as a vector setting the upward direction of the image asthe upward direction of the camera 202. When the gazing pointcoordinates is required to be calculated, the calculation is instructed,and the calculation result of the gazing point coordinates can beacquired at S8B.

The control unit 103 instructs the calculation unit 104 to calculate thecamera coordinates, and acquires a calculation result of the cameracoordinates (S9B). The camera coordinates is calculated based on theviewpoint coordinates calculated at step S6B, the display magnificationcalculated at step S7B, and the gazing point coordinates.

The display magnification causes a change of the angle of view “α” inaccordance with the magnification level, and causes a change of focaldistance determining a distance from the gazing point to the camera 202depending on a value of the angle of view “α”. Therefore, when thedistance from the gazing point to the camera 202 is calculated based onthe display magnification, the camera coordinates can be calculated, forexample, based on the viewing direction of the camera 202 determined bythe viewpoint coordinates, and the calculated distance. The abovedescribed calculation method is just one example. As long as the cameraUP and the camera coordinates can be calculated, other methods can beused.

Since the processing from S10B to S12B is same as the processing from S6to S8 illustrated in FIG. 13, the description of S10B to S12B isomitted.

When the parameter is registered (SB10), the viewpoint management unit105 registers each value in the viewpoint information management table.The viewpoint information management table of the fourth embodimentincludes a field for storing the selected projection type, a field forstoring the camera UP, a field for storing the camera coordinates, and afield for storing the gazing point coordinates, in which the viewpointmanagement unit 105 registers the selected projection type and thecalculated values in the respective fields. As to the angle of view “α”,the angle of view “α” can be stored as the display magnification, whichis the zoom magnification with respect to the initial angle of view “α”,or the value of the angle of view “α” can be stored separately from thedisplay magnification.

Each time the user performs the viewpoint registration operation, theprocessing of from S1B to S12B is repeatedly executed, in which theviewpoints designated by the user are registered in the viewpointmanagement table based on the designation order of the registeredviewpoints.

Hereinafter, a description is given of a transition of viewpoints when apreview is played for two registered viewpoints set with differentprojection types with reference to FIGS. 29 to 31.

FIG. 29 illustrates a scheme of transition of viewpoints from theviewpoint 1 set with the first projection type to the viewpoint 2 setwith the second projection type. In an example case of FIG. 29A, astatus (A) is same as the first projection type illustrated in FIG. 24A,and a status (C) is same as the second projection type illustrated inFIG. 25A. A status (B) indicates a status that is at the middle of thetransition of viewpoints. In example cases of FIGS. 29 to 31, theviewpoints transit along the transition path from the viewpoint 1 at thestatus (A) to the viewpoint 2 at the status (C), in which a viewpoint atthe status (B) is at any intermediate position between the viewpoint 1and the viewpoint 2. In order to simplify the illustration anddescription, the viewpoint at the status (B) is set at the middleposition between the viewpoint 1 and the viewpoint 2. As abovedescribed, each viewpoint corresponds to each frame, which is eachpartial image, configuring one animation. The number of framesconfiguring the one animation can be set any numbers. In example casesof FIGS. 29 to 31, it is assumed that one animation includes threeframes respectively corresponding the viewpoint 1 at the status (A), theviewpoint at the status (B), and the viewpoint 2 at the status (B).

When the status transits from the status (A) to the status (C), thegazing point is same (i.e., origin 201) for both of the status (A) andthe status (C) while the angle of view “α”, the camera coordinates, andthe camera UP are changed between the status (A) and the status (C)Therefore, the viewpoints transit along a transition path interpolatingbetween the viewpoints, and the angle of view “α”, the cameracoordinates, and the camera UP are also transited by interpolatingbetween the viewpoints.

Since the parameters set for the status (A) and the parameters set forthe status (C) are respectively same as the parameters illustrated inFIG. 24B and FIG. 25B, the parameters are not described. The viewpointcontrol unit 106 calculates an angle of view “α” during the transitionbased on the angle of view “α” at the status (A) and the angle of view“α” at the status (C). Specifically, the viewpoint control unit 106calculates a shift value of the angle of view “α” of each frame based ona difference between the angle of view “α” at the viewpoint 1 and theangle of view “α” at the viewpoint 2.

As illustrated in FIG. 29B, since the angle of view “α” at the viewpoint1 (status (A)) is 120 degrees, and the angle of view “α” at theviewpoint 2 (status (C)) is 170 degrees, the difference of the angle ofview “α” between the viewpoint 1 and the viewpoint 2 becomes 50 degrees.The angle of view “α” at the status (B) during the transition iscalculated by adding the shift value of the angle of view “α” from theviewpoint 1 to the middle position, which is calculated based on thedifference of the angle of view “α” and a frame position at the status(B), to the angle of view “α” at the viewpoint 1. In this example case,the angle of 25 degrees, which is a half of the difference of 50 degreesbetween the angle of view “α” at the status (A) and the angle of view“α” at the camera coordinates at the status (C) is added to the angleview of 120 degrees at the viewpoint 1 to calculate 145 degrees as theangle of view “α” at the status (B).

The viewpoint control unit 106 calculates the camera coordinates duringthe transition based on the camera coordinates at the status (A) and thecamera coordinates at the status (C). Specifically, the viewpointcontrol unit 106 calculates the shift value of the camera coordinatesfor each frame based on the distance between the camera coordinates atthe viewpoint 1 and the camera coordinates at the viewpoint 2.

The camera coordinates at the status (B) during the transition iscalculated by adding the shift value of the camera coordinates from theviewpoint 1 to the middle position, which is calculated based on thedistance between the viewpoint 1 and the viewpoint 2 and a frameposition at the status (B), to the camera coordinates at theviewpoint 1. In this example case, the distance in the Y axis directionbetween the viewpoint 1 and the viewpoint 2 is set one (1), and thedistance in the Z axis direction between the viewpoint 1 and theviewpoint 2 is set 1.8. The camera coordinates at the status (B) iscalculated by adding 0.5 (i.e., half of 1) and 0.9 (i.e., half of 1.8)respectively to the y coordinate and the z coordinate of the cameracoordinates of (0, 0, −1.8) at the viewpoint 1 to obtain the cameracoordinates of (0, 0.5, −0.9) at the status (B) In this example case,the distance in the Y axis direction and the distance in the Z axisdirection are respectively added with the values because the viewpointstransits to the direction that the value becomes greater in both of they coordinate and the z coordinate along a direction from the viewpoint 1to the viewpoint 2.

The viewpoint control unit 106 calculates the camera UP at the status(B) during the transition based on the camera UP at the status (A) andthe camera UP at the status (C). Specifically, the viewpoint controlunit 106 calculates a shift value of the camera UP for each frame basedon the rotation angle obtained from the camera UP at the viewpoint 1 andthe camera UP at the viewpoint 2.

The camera UP at the status (B) during the transition is calculated byadding the shift value of the camera UP from the viewpoint 1 to themiddle position, which is calculated based on the rotation angle and aframe position at the status (B), to the camera UP at the viewpoint 1,by subtracting the shift value of the camera UP from the viewpoint 1 tothe middle position from the camera UP at the viewpoint 1, or by bothadding and subtracting the shift value of the camera UP for the cameraUP at the viewpoint 1.

As illustrated in FIG. 29B, since the camera UP at the viewpoint 1 is(0, 1, 0) and the camera UP at the viewpoint 2 is (0, 0, 1), the upwarddirection of the camera 202 is rotated for 90 degrees from the Y axis tothe Z axis direction. In this example case, the camera UP at the status(B) during the transition is calculated by subtracting 0.5 for the ycoordinate and adding 0.5 for the z coordinate for the camera UP of (0,1, 0) at the viewpoint 1 to obtain the camera UP of (0, 0.5, 0.5) at thestatus (B), in which 0.5 corresponds to an angle of 45 degrees that is ahalf of the rotation angle of 90 degrees. In this example case, the ycoordinate receives the subtraction and the z coordinate receives theaddition because the viewpoints transit to the direction that the valuebecomes smaller for the y coordinate, and the value becomes greater forthe z coordinate.

FIG. 30 illustrates a scheme of transition of viewpoints from theviewpoint 1 set with the second projection type to the viewpoint 2 setwith the third projection type. In an example case of FIG. 30(a), astatus (A) is same as the second projection type illustrated in FIG.25A, and a status (C) is same as the third projection type illustratedin FIG. 26A. A status (B) indicates a status that is at the middle ofthe transition of viewpoints.

When the status transits from the status (A) to the status (C), all ofthe angle of view “α”, the camera coordinates, the camera UP, and thegazing point coordinates are changed between the status (A) and thestatus (C). Therefore, when the viewpoints transit along a transitionpath interpolating between the viewpoints, the angle of view “α”, thecamera coordinates, the camera UP, and the gazing point coordinates arealso transited by interpolating between the viewpoints.

Since the parameters se for the status (A) and the parameters set forthe status (C) are respectively same as the parameters illustrated inFIG. 25B and FIG. 26B, the parameters are not described. The viewpointcontrol unit 106 calculates an angle of view “α” during the transitionbased on the angle of view “α” at the status (A) and the angle of view“α” at the status (C) using the method described with reference to FIG.29.

In this example case, the angle of view “α” at the status (B) during thetransition is calculated by adding the shift value of 17.5 degrees,which is a half of the difference of 35 degrees between the angle ofview “α” at the status (A) and the angle of view “α” at the status (C),to the angle of view “α” of 170 degrees at the viewpoint 1 to calculate187.5 degrees as the angle of view “α” at the status (B).

The viewpoint control unit 106 calculates the camera coordinates duringthe transition based on the camera coordinates at the status (A) and thecamera coordinates at the status (C) using the method described withreference to FIG. 29.

In this example case, the camera coordinates at the status (B) iscalculated by subtracting the shift value of 0.5 for the y coordinatefrom the camera coordinates of (0, 1, 0) at the viewpoint 1 to obtainthe camera coordinates of (0, 0.5, 0) at the status (B). In this examplecase, the total sum of the shift value of the distance in the Y axisdirection between the viewpoint 1 and the viewpoint 2 is one (1), and0.5 is a half of 1.

The viewpoint control unit 106 calculates the camera UP at the status(B) during the transition based on the camera UP at the status (A) andthe camera UP at the status (C) using the method described withreference to FIG. 29.

As illustrated in FIG. 30B, since the camera UP at the viewpoint 1 is(0, 0, 1) and the camera UP at the viewpoint 2 is (0, 1, 0), the upwarddirection of the camera 202 is rotated for 90 degrees from the Z axis tothe Y axis direction. In this example case, the camera UP at the status(B) during the transition is calculated by adding the shift value of 0.5for the y coordinate, and subtracting the shift value of 0.5 for the zcoordinate for the camera UP of (0, 0, 1) at the status (A) to obtainthe camera UP of (0, 0.5, 0.5) at the status (B), in which 0.5corresponds to an angle of 45 degrees that is a half of the rotationangle of 90 degrees between the status (A) and the status (C).

The viewpoint control unit 106 calculates the gazing point coordinatesat the status (B) during the transition based on the gazing pointcoordinates at the status (A) and the gazing point coordinates at thestatus (C). Specifically, the viewpoint control unit 106 calculates theshift value of the gazing point for each frame based on the distancebetween a gazing point 1 at the viewpoint 1 and a gazing point 2 at thesecond viewpoint 2.

The gazing point coordinates at the status (B) during the transition iscalculated by adding the shift value of 0.5 for the z coordinate to thegazing point coordinates of (0, 0, 0) at the viewpoint 1 to obtain thegazing point coordinates of (0, 0, 0.5) at the status (B). The shiftvalue of 0.5 is a half of the distance of one (1) in the Z-axisdirection between the gazing point at the viewpoint 1 and the gazingpoint at the viewpoint 2.

FIG. 31 illustrates a scheme of transition of viewpoints from theviewpoint 1 set with the third projection type to the viewpoint 2 setwith the first projection type. In an example case of FIG. 31A, a status(A) is same as the third projection type illustrated in FIG. 26A, and astatus (C) is same as the first projection type illustrated in FIG. 24A.A status (B) indicates a status that is at the middle of the transitionof viewpoints.

When the status transits from the status (A) to the status (C), thecamera UP is same for both of the status (A) and the status (C) whilethe angle of view “α”, the camera coordinates, and the gazing pointcoordinates are changed between the status (A) and the status (C)Therefore, when the viewpoints transit along a transition pathinterpolating between the viewpoints, the angle of view “α”, the cameracoordinates, and the gazing point coordinates are also transited byinterpolating between the viewpoints.

Since the parameters set for the status (A) and the parameters set forthe status (C) are respectively same as the parameters illustrated inFIG. 26B and FIG. 24B, the parameters are not described. The viewpointcontrol unit 106 calculates an angle of view “α” during the transitionbased on the angle of view “α” at the status (A) and the angle of view“α” at the status (C) using the method described with reference to FIG.29.

In this example case, the angle of view “α” at the status (B) during thetransition is calculated by subtracting the shift value of 42.5 degrees,which is a half of the difference of 85 degrees between the angle ofview “α” at the status (A) and the angle of view “α” at the status (C),from the angle of view “α” of 205 degrees at the viewpoint 1 tocalculate 162.5 degrees as the angle of view “α” at the status (B).

The viewpoint control unit 106 calculates the camera coordinates duringthe transition based on the camera coordinates at the status (A) and thecamera coordinates at the status (C) using the method described withreference to FIG. 29.

In this example case, the camera coordinates at the status (B) iscalculated by subtracting the shift value of 0.9 for the y coordinatefrom the camera coordinates of (0, 0, 0) at the viewpoint 1 to obtainthe camera coordinates of (0, 0, −0.9) at the status (B) In this examplecase, the total sum of the shift value of the distance in the Z axisdirection between the viewpoint 1 and the viewpoint 2 is 1.8, and 0.9 isa half of 1.8.

The viewpoint control unit 106 calculates the gazing point coordinatesat the status (B) during the transition based on the gazing pointcoordinates at the status (A) and the gazing point coordinates at thestatus (C) using the method described with reference to FIG. 30.

In this example case, the gazing point coordinates at the status (B)during the transition is calculated by subtracting the shift value of0.5 for the z coordinate from the gazing point coordinates of (0, 0, 1)at the viewpoint 1 to obtain the gazing point coordinates of (0, 0, 0.5)at the status (B) In this example case, the total sum of the shift valueof the distance in the Z-axis direction between the gazing point at theviewpoint 1 and the gazing point at the viewpoint 2 is one (1), and 0.5is a half of one.

With reference to FIGS. 29 to 31, the transition from the firstprojection type to the second projection type, the transition from thesecond projection type to the third projection type, and the transitionfrom the third projection type to the first projection type aredescribed, but the transition patterns are not limited thereto. Forexample, a transition from the second projection type to the firstprojection type, a transition from the third projection to the secondprojection type, and a transition from the first projection type to thethird projection type can be performed using the above described method,in which a partial image is generated based on a change of at least anyone of the angle of view “α”, the camera coordinates, the camera UP, andthe gazing point coordinates for each frame, with which a smoothtransition of the viewpoints, the gazing points, the camera UP and thezoom-in and/or zoom-out of the images can be performed when playing ananimation. Further, by changing the projection type, a dynamic imageexpression using the full-view spherical image can be achieved.

Hereinafter, a description is given of processing that is performed whenplaying the preview by the information processing apparatus 100 withreference to a sequence diagram illustrated in FIG. 32. The processingof the preview playing of FIG. 32 is almost same as the processingillustrated in FIG. 15 described as the first embodiment exceptprocessing at S5C and S7C, and thereby the processing at S5C and S7C aredescribed.

At SSC, the viewpoint control unit 106 controls the transition ofviewpoints along a transition path interpolating between the registeredfirst viewpoint (i.e., start viewpoint) and the registered secondviewpoint (i.e., end viewpoint). In this control process at step S5C,the viewpoint control unit 106 calculates the shift value of theviewpoint for each frame using the above described method, and alsocalculates the shift value of the angle of view “α”, the shift value ofthe camera coordinates, the shift value of the camera UP, the shiftvalue of the gazing point coordinates for each frame based on aprojection type registered for the start viewpoint and a projection typeregistered for the end viewpoint.

At S7C, the viewpoint control unit 106 repeatedly reports the shiftvalue of the viewpoint for each frame, the shift value of the angle ofview “α”, the shift value of the camera coordinates, the shift value ofthe camera UP, the shift value of the gazing point coordinates for eachframe to the control unit 103 based on the projection type registeredfor the start viewpoint and the projection type registered for the endviewpoint until a value obtained by multiplying a frame period to thenumber of frames exceeds the transition time. At this timing, the shiftdirection such as addition and subtraction direction can be alsoreported.

With this configuration, by changing the projection types or modes setfor each of the registered viewpoints, an image expression of thefull-view spherical image can be performed with various patters, whichcannot be expressed by changing the viewpoints and the angle of view “α”alone.

Further, in the fourth embodiment, a user can set a viewpoint transitiondirection, which is the transition direction of the viewpoints, as aviewpoint parameter similar to the second embodiment, and the viewpointscan be transited along the set viewpoint transition direction. Further,in the fourth embodiment, the animation can be performed automatically,the animation can be cropped, and the animation can be synthesized withthe background audio and played using the preset data described in thethird embodiment.

As to the above described information processing apparatus of theembodiments in this disclosure, the information processing apparatus canpresent or display an area of interest in a full-view spherical imageeasily while reducing time and effort of a user operation.

The above-described functions of the embodiments can be implemented byexecuting one or more programs, written in C, C++, C #, Java (registeredtrademark), and the one or more programs can be stored in any storagemedium, carrier medium, carrier means, or digital data carrier forstoring processor readable code such as a flexible disk, a compact diskread only memory (CD-ROM), a digital versatile disk read only memory(DVD-ROM), DVD recording only/rewritable (DVD-R/RW), electricallyerasable and programmable read only memory (EEPROM), erasableprogrammable read only memory (EPROM), a memory card or stick such asUSB memory, a memory chip, a mini disk (MD), a magneto optical disc(MO), and distributed as a storage medium such as a hard disk, a CD-ROM,an MO, a DVD, a flexible disk, EEPROM, EPROM, and further, the one ormore programs can be stored in other devices, from which the one or moreprograms can be transmitted through a network.

Numerous additional modifications and variations are possible in lightof the above teachings. It is therefore to be understood that within thescope of the appended claims, the disclosure of the present inventionmay be practiced otherwise than as specifically described herein. Forexample, elements and/or features of different illustrative embodimentsmay be combined with each other and/or substituted for each other withinthe scope of this disclosure and appended claims.

As can be appreciated by those skilled in the computer arts, thisinvention may be implemented as convenient using a conventionalgeneral-purpose digital computer programmed according to the teachingsof the present specification. Appropriate software coding can readily beprepared by skilled programmers based on the teachings of the presentdisclosure, as will be apparent to those skilled in the software arts.The present invention may also be implemented by the preparation ofapplication-specific integrated circuits or by interconnecting anappropriate network of conventional component circuits, as will bereadily apparent to those skilled in the relevant art.

Each of the functions of the described embodiments may be implemented byone or more processing circuits. A processing circuit includes aprogrammed processor. A processing circuit also includes devices such asan application specific integrated circuit (ASIC) and conventionalcircuit components arranged to perform the recited functions.

What is claimed is:
 1. An information processing apparatus comprising:circuitry to register at least a first viewpoint and a second viewpointdesignated next to the first viewpoint as viewpoints in a full-viewspherical image with an order designated by a user; set a transitionpath of the viewpoints from the first viewpoint to the second viewpointby interpolating between the first viewpoint and the second viewpoint inthe full-view spherical image; generate a first partial image having acenter that matches the first viewpoint, and a second partial imagehaving a center that matches the second viewpoint; and play animation bysequentially displaying the first partial image and the second partialimage with the order designated by the user while transiting theviewpoints from the first viewpoint to the second viewpoint along theset transition path.
 2. The information processing apparatus of claim 1,wherein the circuitry registers the first viewpoint and a first angle ofview designated by the user in association with each other, and thesecond viewpoint and a second angle of view designated by the user inassociation with each other, wherein the circuitry generates the firstpartial image based on the first angle of view associated with the firstviewpoint designated by the user, and the second partial image based onthe second angle of view associated with the second viewpoint designatedby the user.
 3. The information processing apparatus of claim 1, whereinthe circuitry registers at least one of a first transition speed and afirst transition time designated by the user in association with thefirst viewpoint, wherein the circuitry transits the viewpoints from thefirst viewpoint to the second viewpoint along the set transition path byapplying the at least one of the first transition speed and the firsttransition time.
 4. The information processing apparatus of claim 1,wherein the circuitry registers a first transition direction designatedby the user in association with the first viewpoint, wherein thecircuitry transits the viewpoints from the first viewpoint to the secondviewpoint along the transition path set by the first transitiondirection.
 5. The information processing apparatus of claim 1, whereinthe circuitry converts the generated first partial image and the secondpartial image into movie image data.
 6. The information processingapparatus of claim 5, further comprising a display to display a firsticon corresponding to the first viewpoint designated by the user, and asecond icon corresponding to the second viewpoint designated by the userrespectively as a first thumbnail image and a second thumbnail image. 7.The information processing apparatus of claim 6, wherein the displayreceives editing of the first viewpoint and the second viewpointdesignated by the user, and information registered in association withthe first viewpoint and the second viewpoint.
 8. The informationprocessing apparatus of claim 6, wherein the display displays one ormore selections of background audio, and the circuitry synthesizes themovie image data with the background audio selected by the user.
 9. Theinformation processing apparatus of o claim 6, wherein the displaydisplays one or more selections of crop regions, and the circuitry playsthe animation sequentially displaying the first partial image and thesecond partial while cropping the animation by applying the crop regionselected by the user.
 10. The information processing apparatus of claim6, further comprising a memory to store one or more preset dataregistering the first viewpoint and the second viewpoint in thefull-view spherical image with the order designated by the user, whereinthe display displays the one or more preset data, wherein the circuitrytransits the viewpoints from the first viewpoint to the secondviewpoint, next to the first viewpoint, registered in the preset dataselected by the user, along the transition path set by applying thepreset data selected by the user.
 11. The information processingapparatus of claim 10, wherein the preset data includes a first angle ofview registered in association with the first viewpoint, and a secondangle of view registered in association with the second viewpoint,wherein the circuitry generates the first partial image based on thefirst angle of view associated with the first viewpoint, and the secondpartial image based on the second angle of view associated with thesecond viewpoint registered in the preset data selected by the user. 12.The information processing apparatus of claim 11, wherein the displayreceives a change operation of the second viewpoint registered as an endviewpoint registered in the preset data selected by the user, and achange operation of the second angle of view associated with the secondviewpoint registered as the end viewpoint in the preset data selected bythe user, wherein when the second viewpoint registered as the endviewpoint is changed, the circuitry transits the viewpoints from thefirst viewpoint to the changed second viewpoint along the transitionpath interpolating between the first viewpoint and the changed secondviewpoint, wherein when the second angle of view associated with thesecond viewpoint is changed, the circuitry generates the second partialimage based on the changed second angle of view.
 13. The informationprocessing apparatus of claim 6, wherein the display receives aselection of a projection mode used for projecting each of the firstpartial image and the second partial image on the display, wherein thecircuitry registers the first viewpoint and first projection informationused for projecting the first partial image by a first projection modein association with each other, and registers the second viewpoint andsecond projection information used for projecting the second partialimage by a second projection mode in association with each other,wherein the circuitry generates the first partial image based on thefirst projection information associated with the first viewpointdesignated by the user, and the second partial image based on the secondprojection information associated with the second viewpoint designatedby the user.
 14. The information processing apparatus of o claim 13,wherein the circuitry registers the first projection information of thefirst projection mode in association with the first viewpoint, and thesecond projection information of the second projection mode inassociation with the second viewpoint, wherein the circuitry generatesthe first partial image having the center that matches the firstviewpoint based on the first projection information, and the secondpartial image having the center that matches the second viewpoint basedon the second projection information, and the circuitry transits theviewpoints from the first partial image, generated by applying the firstprojection mode, to the second partial image, generated by applying thesecond projection mode.
 15. The information processing apparatus ofclaim 13, wherein the circuitry maps the full-view spherical image on asurface of a virtual three dimensional object, and obtains the firstpartial image and the second partial image as images captured by avirtual image capture, wherein the circuitry registers information of anangle of view of the virtual image capture, information of a position ofthe virtual image capture, information indicating an upward direction ofthe virtual image capture, and information indicating a position of agazing point of the virtual image capture as the first projectioninformation to be used when the virtual image capture captures the firstpartial image, wherein the circuitry registers information of an angleof view of the virtual image capture, information of a position of thevirtual image capture, information indicating a upward direction of thevirtual image capture, and information indicating a position of a gazingpoint of the virtual image capture as the second projection informationto be used when the virtual image capture captures the second partialimage.
 16. The information processing apparatus of claim 15, wherein thecircuitry generates the information of the angle of view of the virtualimage capture, the information of the position of the virtual imagecapture, the information indicating the upward direction of the virtualimage capture, and the information indicating the position of the gazingpoint of the virtual image capture based on the first viewpoint and thesecond viewpoint respectively designated by the user, the angle of viewof the first viewpoint and the second viewpoint respectively designatedby the user, information of a upward direction of the full-viewspherical image designated by the user when the user designates thefirst viewpoint and the second viewpoint, and the projection moderespectively selected for the first viewpoint and the second projectionmode.
 17. A method of processing information comprising: registering atleast a first viewpoint and a second viewpoint designated next to thefirst viewpoint as viewpoints in a full-view spherical image with anorder designated by a user; setting a transition path of the viewpointsfrom the first viewpoint to the second viewpoint by interpolatingbetween the first viewpoint and the second viewpoint in the full-viewspherical image; generating a first partial image having a center thatmatches the first viewpoint, and a second partial image having a centerthat matches the second viewpoint; and playing animation by sequentiallydisplaying the first partial image and the second partial image with theorder designated by the user while transiting the viewpoints from thefirst viewpoint to the second viewpoint along the set transition path.18. A non-transitory storage medium storing one or more instructionsthat, when executed by one or more processors, cause the one or moreprocessors to execute a method of processing information comprising:registering at least a first viewpoint and a second viewpoint designatednext to the first viewpoint as viewpoints in a full-view spherical imagewith an order designated by a user; setting a transition path of theviewpoints from the first viewpoint to the second viewpoint byinterpolating between the first viewpoint and the second viewpoint inthe full-view spherical image; generating a first partial image having acenter that matches the first viewpoint, and a second partial imagehaving a center that matches the second viewpoint; and playing animationby sequentially displaying the first partial image and the secondpartial image with the order designated by the user while transiting theviewpoints from the first viewpoint to the second viewpoint along theset transition path.